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Overview 

• Introduction 

• Video 

• Mixing Music for VR 

• Implementation 
. Q & A 
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Introduction 

Erik Desiderio 

• Composer for This Means War! 
by Kabam, Emmy Award Show, 
Entourage , Duck Dynasty 

• Music Producer for Immortals, 
The Borgias, Raising Hope 

• Model Ship Builder 

• Last Name rhymes with "Car 
Stereo" 
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Video 



http://www.erikdesiderio.com/adventure-time-vr-video/ 
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Music 

• Instrumentation 

• Game Music vs. Show Music 

• Music for game is unique. No themes from show 

• Show has ukulele and 8 bit sounds 

• Game has natural sound, real instruments 

. Clarinet, Banjo, Fretless Bass, Accordion, Harmonica, 
Cornet, Whistling, etc. No Ukulele! 

• Most sound effects came from show 




ink. 
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Project Details 

• Adventure Time: Magic Man's Head Games 

• Gear VR, Oculus, Vive, PSVR 

• Turbo Button for Cartoon Network 

• Met the game designer in a bar 

• Worked on 6 games together, from small 
mobile games to VR 


in 
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Turbo Button 


SMS Racing 
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Turbo Button 


Floor Plan 
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Mixing Music for VR 

• Headphone Mixing 

• Correct Type of Headphones 

• Music Localization 

• Binaural Sound Processing 



Do not use! 



1 1 
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Headphone Mixing 

• If you are mixing through speakers, stop 
You're doing it wrong. 

• Use speakers for soundtrack album 



iiik 
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Headphone Mix v. Speaker Mix 

• Instrument panning is exaggerated in 
headphones 

• Reverb is less subtle in headphones 

• Often 5k notch in headphones. Speakers attempt 
to be flat 
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Frequency Response 
Top - Compensated and Averaged 
Bottom - Raw Data for Five Headphone Positions 
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Use Stereo Headphones 

• Using surround modeling headphones 
may produce unintended effects 

• Surround processing =£ Binaural 
processing 


GDC 




GAME DEVELOPERS CONFERENCE March 14-18, 2016 ■ Expo: March 16-18, 2016 #G0C16 


Oculus Rift Built in Headphones 



Oculus Headset All Other VR Headsets 
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Music Localization 

• Localizing = controlling where the listener 
perceives a sound to come from 

• Not dialog localization, which refers to taking a 
game and putting it into a different language 




Localizing Music v. SFX 


• Independent of 
game environment/ 
head location 

• Reverb not in game 
environment, can 
be pre-mixed 


• SFX must be 
correlated to 
objects/actions 

• SFX reverb must be 
generated in game, 
realistic 
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2D Mix Localization whistlel 

Banjo Bass Accordion 
Trpt 3 Trpt 1 Trpt 2 


Clarinet Bass Clarinet 


Congas 


Hi Hat 


Shaker 1 


Whistle 4 



Shaker 2 
Triangle 
Whistle 3 
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2D Mix Localization 


Banjo 


Accordion 
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<> 


2D Mix Localization 


Trumpet 1 


Trumpet 3 Trumpet 2 
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2D Mix Localization 

Whistling 1 
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Mix Width Tips 

• Less instruments, narrower mix 

• More instruments, wider mix 

• Still keep melody in center of mix 
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Music Localization Techniques 

• #1 Simple Panning 

• This will increase volume on one channel, decrease 
volume on the other channel making the source 
appear from that side 




iQJ 

pan 100t 


Pan Left 


Pan Center 


Pan Right 
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Music Localization Techniques 

• #2 Delay one channel of a duplicated mono 
file by 1-2 milliseconds 

• Step one, start with mono audio file panned center 


mj 04 

pan ► 0 < 




Center 
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Music Localization Techniques 

• #2 Delay one channel of a duplicated mono 
file by 1-2 milliseconds 

• Step two, duplicate and pan hard left and right 

■ 1 
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Music Localization Techniques 


• #2 Delay one channel of a duplicated mono 
file by 1-2 milliseconds 


• Step three, nudge one file 1-2 ms to the right 



^ -7-1 dB 






GAME DEVELOPERS CONFERENCE March 14-18, 2016 ’ Expo: March 16-18, 2016 #GDC16 


GDC 


<> 


Music Localization Techniques 

• #2 Delay one channel of a duplicated mono 
file by 1-2 milliseconds 

• You can copy them to a stereo track panned center 
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Music Localization Techniques 

• #3 Add reverb early reflections or EQ to 
side 



• These 3 techniques affect horizontal 
placement, Still In 2DM! 
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Binaural Sound Processing 




• Good for vertical localization, adds 3D in 
headphones 
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What is 3D Audio? 

• Not limited to horizontal pla 

• Height (above/below) 

• Depth (near/far) 

• Around (360 degrees) 
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Creative Mixing in 3D 

• Wow, now I can mix in 3D, Now what? 

• Approaches for vertical: 

• Bass on bottom, treble on top 

• "Wedding Cake" Approach 

• Cake: Bass & Drums on bottom 

• Icing: Melody higher up 

• Figure/Candles: Synths, Shakers 
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Binaural Plug-Ins 

• Logic Sequencer Plugin 

• Processes audio before it is in 
game engine 

• Doesn't tax game cpu 
resources, it's free! 


o 

'Vokw 




— Pucks 


Panning plane 



jtton 


Extended parameters 

Click the disclosure 
triangle to reveal these 
extended parameters. 
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Binaural Plug-Ins 

• Logic Sequencer Plugin 

• Processes audio before it is in 
game engine 

• Doesn't tax game cpu 
resources, it's free! 




Puck 


Panning plar 


— 3D image 
J- Mode buttons 


— Size field 

— Doppler button 


Extended parameters 

Click the disclosure 
triangle to reveal these 
extended parameters. 


HI 
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Binaural Plug-Ins 

• Logic Sequencer Plugin 

• Processes audio before it is in 
game engine 

• Doesn't tax game cpu 
resources, it's free! 


Pucks 


— Panning plane 

— 3D image 

\- Mode buttons 



Size field 
Doppler button 


- Extended parameters 

Click the disclosure 
triangle to reveal these 
extended parameters. 


OMKTtCfl 
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3 rd Party Binaural Plugins 

• 3Dception by Two Big Ears 

• Real Space by 3D Audio 

• Use in Unity, Fmod, Wwise 


r n 

30CEPTiON 
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Oculus Audio SDK 
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Limitations of Binaural Processing 


• Only for use on headphones 

• Models generic human head 
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Limitations of Binaural Processing 


• Plane/Cone of confusion 
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Surround v. Binaural 

• Surround is great for speakers, still 2D 

• Binaural is great for headphones, 3D 
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Implementation 

•Game was created in Unity 4.6 
•MasterAudio plugin for Unity 
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Adaptive Music Method 1 

• 2 audio files: 

• 1 Intro Mix with clarinet light percussion and banjo 
•2 Full Mix Loop all instruments 


• Multiple start points for track 
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Adaptive Music Method 2 

• Start full mix loop after a certain point 
• 2 audio files: 

• 1 Intro Mix loop, light mix 
•2 Full Mix loop, all instruments 
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Adaptive Music Method 3 

• Fade in additional layers 

• Multiple audio files play at the same time: 

• 1 "Exploration" loop, clarinet, banjo, synths 
•2 Percussion Layer 
•3 Melody and heavy drums 
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Adaptive Audio Implementation 

• Additional Techniques 

• Pitch drop on character fall 

• Music volume lower during dialog 

• Smooth fade out on level completion 

• Spacial processing and reverb in sequencer 


in 
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Adaptive Audio Implementation 

• Changing Emotion with Music Mix 
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Adventure Time Binaural Mix 

• Listen to it later in quiet room with 
Headphones on: 

• erikdesiderio.com/atb/ 
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Q & A 


• Step up to the mic 

• Ask me anything 

• Please fill out survey 

• Questions/insults: 
erik@erikdesiderio.com 






/ 4. . /• 




Wrap up Room: 3022 


Binaural Mix: erikdesiderio.com/atb/ 
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